Lagrange Multipliers and Optimality

Lagrange multipliers used to be viewed as auxiliary variables introduced in a problem of constrained minimization in order to write first-order optimality conditions formally as a system of equations. Modern applications, with their emphasis on numerical methods and more complicated side conditions than equations, have demanded deeper understanding of the concept and how it fits into a larger theoretical picture. A major line of research has been the nonsmooth geometry of one-sided tangent and normal vectors to the set of points satisfying the given constraints. Another has been the game-theoretic role of multiplier vectors as solutions to a dual problem. Interpretations as generalized derivatives of the optimal value with respect to problem parameters have also been explored. Lagrange multipliers are now being seen as arising from a general rule for the subdifferentiation of a nonsmooth objective function which allows black-and-white constraints to be replaced by penalty expressions. This paper traces such themes in the current theory of Lagrange multipliers, providing along the way a free-standing exposition of basic nonsmooth analysis as motivated by and applied to this subject. 1. Optimization problems. Any problem of optimization concerns the minimization of some real-valued, or possibly extended-real-valued, function f 0 over some set C; maximization can be converted to minimization by a change of sign. For problems in finitely many " continuous " variables, which we concentrate on here, C is a subset of lR n and may be specified by a number of side conditions, called constraints, on x = (x 1 ,. .. , x n). Its elements are called the feasible solutions to the problem, in contrast to the optimal solutions where the minimum of f 0 relative to C is actually attained in a global or local sense. Equality constraints f i (x) = 0 and inequality constraints f i (x) ≤ 0 are most common in describing feasible solutions, but other side conditions, like the attainability of x as a state taken on by a controlled dynamical system, are encountered too. Such further conditions can be indicated abstractly by a requirement x ∈ X with X ⊂ lR n. This notation can be convenient also in representing simple conditions for which the explicit introduction of a constraint function f i would be cumbersome, for instance sign restrictions or upper or lower bounds on the components x j of x. In a standard formulation of optimization from this point …